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A  tail  pr  obabi  I  ity  expresses  the  likelihood  that  a 
random  variable  will  exceed  some  specified  threshold.  Tail 
probabilities  are  important  to  system  designers  because  values 
exceeding  certain  thresholds  may  represent  unacceptable  or  even 
catastrophic  system  behaviors.  This  paper  investigates  the 
problem  of  obtaining  the  tail  probability  of  a  random  variable 
whose  transform  is  given  explicitly,  but  whose  distribution 
is  not  necessarily  known.  This  problem  typically  arises  when 
the  random  variable  of  interest  is  a  function  of  given  "elemen¬ 
tary**  system  variables.  Applications  include: 

(1)  Reliability  analysis,  where  the  probability  of  system 
failure,  a  function  of  known  individual  component  reliabilities, 
must  not  exceed  a  given  tolerance,  and 

(2)  Queuing  systems,  where  the  probability  of  buffer  over¬ 
flow  or  excessive  waiting  time  is  sought. 

Unfortunately,  tail  probabilities  can  be  difficult  to  com¬ 
pute,  especially  when  the  random  variable’s  distribution  is  not 
available  in  closed  form.  The  standard  "transform  inversion 
formula"  is  an  analytical  expression  (an  integral  over  an  infi¬ 
nite  range),  not  an  algorithm.  Chebyshev  bounds  (on  the  tail 
probability)  are  easily  obtained  from  a  transform,  but  are  rarely 
tight  enough  to  be  useful. 

Various  methods  have  been  proposed  for  the  related  problem 
of  recovering  the  density  of  a  random  variable  from  its 
transform.  But  computing  a  tail  probability  by  first  computing 
the  density  entails  new  difficulties:  the  density  may  not  exist; 
many  density  evaluations  are  needed  to  perform  the  tail  integral. 
Furthermore,  these  methods  are  difficult  to  implement  and  few  in¬ 
clude  numerically  computable  bounds  on  error  and  computational 
effort.  Many  of  the  techniques  surveyed  by  Piessons  (1975)  and 
Piessons  and  Dang  (1976)  involve  solving  large,  ill-conditioned 
systems  of  linear  equations.  Jagerman  (1978)  offered  a  method 
based  on  the  derivatives  of  the  transform.  While  this  paper 
was  in  preparation,  Honig  and  Hirdes  (1984)  published  a  method 
similar  to,  but  weaker  than  ours. 
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In  this  paper,  we  introduce  a  more  direct  algorithm  to 
compute  tail  probabilities  from  transforms.  Our  method  is  both 
faster  and  simpler  than  others  previously  proposed.  It  requires 
virtually  no  memory  and  consists  essentially  of  summing  a  power 
series.  It  also  permits  the  computational  effort  and  worst-case 
error  to  be  specified  in  advance  -  the  smaller  the  accept¬ 
able  error,  the  greater  the  effort  required.  Using  the  methods 
of  computational  complexity  theory  [1],  we  show  that  our  method 
is  a  fully  polynomial  approximation  scheme  and  we  also 
offer  a  strong  argument  that  there  is  no  significantly  better 
algorithm  by  showing  a  formalization  of  the  problem  to  be  NP- 
hard.  Finally,  we  provide  numerical  examples  to  illustrate  the 
manner  in  which  errors  are  prespecified  and  the  ease  with  which 
approximate  solutions  are  obtained. 

Our  method  can  also  be  used  to  obtain,  from  its  transform, 
the  probability  that  a  random  variable  will  fall  within  any  given 
interval  of  the  real  line.  It  can  also  be  modified  Cin  a 
straightforward  way)  to  provide  strict  upper  or  lower  bounds  on 
probabilities,  rather  than  arbitrarily  close  estimates.  However, 
we  have  restricted  our  discussion  to  approximately  computing 
(upper)  tail  probabilities  because  (1)  in  our  experience,  most 
applications  call  for  good  estimates  of  tail  probabilities;  and 

(2)  tail  probabilities  are  small  (typically  between  lO"^  to  10"®) 
and  so  are  especially  vulnerable  to  distortion  by  the  roundoff 
errors  of  floating-point  arithmetic.  Accordingly,  we  will  refer 
to  the  complementary  cumulative  distribution  function  P[X>-1 
as  the  "tail  probability  function." 


1.  THE  PROBLEM  AND  OUR  METHOD  FOR  SOLVING  IT 

Since  most  transforms  are  interchangeable  with  minimal 
computational  effort,  we  will  focus  our  attention  on  the  fami¬ 
liar  (bilateral ]  Laplace  transform,  expressed  as  a  function 
of  the  argument  s,  and  defined  by 

3fx(s)  -  ECe"®*].  (1.1) 

where  X  is  the  random  variable  whose  distribution  is 
"transformed."  It  is  customary  to  view  s  and  Sf(a)  as 
complex  numbers. 


The  random  variable  X  may  be  continuous  or  discrete:  we 
simply  require  that  the  expectation  (1.1)  exists  for  any  imagi¬ 
nary  3  (i.e.,  any  s  whose  real  part  is  zero).  All  distributions 
of  practical  interest  satisfy  this  criterion,  notably  those, 
such  as  the  waiting  time  in  a  queueing  system,  that  are  partly 
continuous  and  partly  discrete,  and  to  which  conventional  methods 
of  transform  inversion  are  inapplicable. 

The  appeal  of  transforms  is  based  on  the  fact  that  many 
simple  operations  on  random  variables  correspond  to  simple 
operations  on  their  transforms,  while  their  distributions  undergo 
more  complicated  transformations.  For  example.  If  Xj 

is  a  sum  of  independent  random  variables,  then 

SJxCs)  -  E[e"*^‘*‘]  -  IT,  E[e‘**‘]  -  IT,  «x,^s).  (1.2) 

that  is,  the  transform  of  X  is  merely  the  product  of  the 
transforms  of  the  components.  X^.  By  way  of  comparison,  the 
density  of  X  is  the  convolution  of  the  components* 
densities,  and  is  much  harder  to  compute. 

Suppose  that  we  are  given  a  procedure  to  compute 
2x(s)  for  any  imaginary  argument  s.  Our  problem  is  to 
compute  the  tail  probability  P[X>A],  where  A  is  a  given 
parameter.  We  propose  to  compute  an  approximate  tail 
probability,  t,  according  to  the  finite  summation 

N  2 

T  -  w  • 

where  im(*]  denotes  the  imaginary  part  of  a  complex  number, 
and 

a  =  ^  p  ,  exp(-Awj)  »  cos(A(o)  -  j  sin(Aw), 

(1.4) 

y  «  exp(-U(i)j)  -  cosdJw)  -  j  sindJw),  w  -  ,  j  -  -"TT  . 

Note  that  r  is  a  function  of  the  transform  2(*)  and 
the  threshold  A,  as  well  as  four  parameters:  L,  U.  D  and  N.  We 
will  show  that  these  parameters  can  be  chosen  to  make  r  as  close 
as  desired  to  the  true  solution  P[X>A1.  The  computational  effort 
is  essentially  determined  by  N.  We  discuss  the  effort  and  error 
associated  with  this  approximation  in  the  two  sections  that 
follow. 


2.  COMPUTATIONAL  REQUIREMENTS 


The  standard  measures  of  computational  effort  are  number  of 
elementary  operations  (addition,  subtraction,  multiplication  or 
division]  performed  and  number  of  memory  "registers"  utilized. 
To  obtain  these  measures,  we  must  explicitly  consider  the  preci¬ 
sion  to  which  operations  are  performed,  and  how  values  of  the 
transform  are  obtained.  To  this  end,  we  introduce  two  technical 
assumptions: 

(Al)  All  operations,  as  well  as  evaluations  of  the 
transform,  are  performed  to  k  significant  decimal  digits. 

(A2)  The  value  of  obtained  in  a  finite  number 

of  operations  that  do  not  depend  on  s. 

The  sum  (1.3]  is  robust  with  respect  to  truncation/roundoff 
errors  since  none  of  its  terms  exceeds  1  in  absolute  value. 
Thus,  direct  evaluation  of  t  according  to  (1.3]  and  (A.l] 

results  in  an  error  of  at  most  *N  10"^ 

There  is,  however,  a  faster  way  to  evaluate  t*.  the  powers 
of  a,  0  and  y  in  (1.3]  may  be  generated  recursively.  Seven 
memory  registers  will  suffice-,  they  contain  (1)  the  cumulative 

sum  to  date,  (2]  ,  (  3  ]  ,  (4-5]  the  complex 

pair  p",  and  (6-7)  the  complex  pair  y”.  Note  that  this 
memory  requirement  is  small  and  does  not  depend  on  N. 

Transform  evaluations  may  be  accelerated  in  a  similar  way  if 
there  holds 

(A. 2*]  may  be  expressed  as  a  finite  number  of 

operations  upon  s  and  functions  of  s  taking  the  form  e*^®™ 
for  some  number  c  and  some  integer  m. 

Clearly  (A. 2’)  (A. 2].  But  if  (A. 2’)  holds,  the  trig¬ 
onometric  functions  in  j^x  computed  once  and  then 

recursively  generated,  as  described  above.  This  substantially 
decreases  the  number  of  operations  required  to  evaluate  the 
transform,  and  it  Increases  the  memory  requirement  only  by  a 
finite  quantity  that  depends  on  but  not  N. 

Since  each  term  in  (1.3]  requires  a  finite  number  of  elemen¬ 
tary  operations  (including  updating  the  powers  of  a,  p  and  y, 
and  evaluating  the  transform),  only  0(N)  operations  are  required 
to  compute  t. 


3.  ERROR  ANALYSIS 

We  now  consider  how  close  t  will  be  to  the  desired  tail 
probability  P[X>A].  Let  Z  be  a  zero  mean,  unit  variance,  normally 
distributed  random  variable  independent  of  X,  and  define 

0  -  P(X-DZ-L  mod  U-L  >  A-L  mod  U-L],  (3.1) 

where  L,  U  and  D  are  given  numbers  such  that  L  <<  A  <<  U  and 
D/(U-L)  <<  1.  Intuitively,  L  and  U  represent  lower  and  upper 
limits  (respectively)  on  the  effective  range  of  X  (that  is, 
P[L  <  X  <  U]  ~  1)  and  D  represents  the  accuracy  to  which  X  and  A 
are  specified.  We  will  view  6  as  an  approximation  to  P{X>A]. 
To  make  6  close  to  P[X>A],  we  must  make  D  very  small  (i.e. 
positive  but  near  zero),  L  of  large  negative  magnitude  and  U  of 
large  positive  magnitude. 

In  Appendix  A,  we  show  that  8  is  exactly  determined  by  the 
infinite  sum 


®  ^  ^  ^  *  im[(p"  -  y")  •  2x(-nwj)]. 


Note  that  (1.3)  contains  only  the  first  N  terms  of  (3.2). 

For  clarity  of  presentation,  we  break  the  error  by  which  t 
and  P{X>AJ  may  differ  into  three  components,  the  accuracy  E,, 
the  precision  Ep,  and  the  truncation  E^.  These  variables  will 
serve  as  error  bounds  in  the  sense  that 

P[X2A+E,»(U-L)1  -  Ep  <  0  <  P(X>A-E,«(U-L)]  ♦  Ep  , 

(3.3) 

0-E|<t<0^E|. 

Loosely  speaking,  (3.3)  states  that  t  must  lie  within  E^  of 
0,  and  0  must  lie  within  Ep  of  a  probability  in  the  range 

from  P{X>A']  to  P(X>A'],  where  A'  differs  from  A  by  less  than  a 
fraction  E,  of  the  effective  range  of  X.  This  is  illustrated 
in  Figure  1.  Accuracy  is  expressed  as  a  fraction  of  the  range 
U-L  of  X,  so  that  E^,  like  the  other  errors,  is  dimensionless. 
(A  fourth  source  of  error,  the  roundoff  caused  by  computing  to  k 
decimal  digits,  was  discussed  in  Section  2.) 


F  I.  - 


The  error  bounds  E.  and  the  parameters  D  and  N  in 
(1.3)-(1.4)  will  be  expressed  as  functions  of  L,  U,  and  three 
additional  variables:  r.  Zp  and  z^.  The  first,  called 

resol ut ion,  specifies  the  desired  accuracy: 


The  second,  Zp,  expresses  the  desired  precision  in  the  sense 
that  Ep  should  be  comparable  to  $(Zp),  where 


♦Cz)  -  PlZ>z]  -  J"  e-*^  dx 


is  the  familiar  tail  function  associated  with  the  normal 
distribution.  The  third,  z^,  similarly  expresses  the  desired 
truncation  error. 

No  closed-form  expression  for  the  integral  (3.5)  exists,  and 
the  tables  in  most  statistics  texts  do  not  extend  into  the  range 
of  interest  to  us  here.  So  we  have  provided  a  graph  of  $  in 
Figure  2.  It  shows,  for  example,  that  z-6  corresponds  to  the 

precision  10~®  . 

The  exact  relationship  between  r,  Zp  and  Z|  and  the 
errors  E,  isgivenby 


Theorem  1.  The  bounds  (3.3)  are  satisfied  when 
E,  -  rzp  , 

Ep  -  max{P[X<L*DZpl.  P(X>U-DZpl)  >  2  ffZp), 


-t  z 


z^  -  2nrN. 

Proof.  See  Appendix  B. 


If  L  and  U  are  sufficiently  far  apart,  (3.7)  and  (3.8)  imply 
that  the  errors  Ep  and  E^  may  be  made  extremely  small  by 

selecting  modest  values  of  Zp  and  Z(  (see  Figure  2).  Speci¬ 
fically,  Ep  and  E^  can  be  made  within  1  (that  is, 

negligible  when  k-digit  decimal  arithmetic  is  used)  by  choosing 
Zp  and  Z|  to  be  0(k).  Moreover,  for  fixed  Zp  ,  (3.6)  shows 

that  the  accuracy  E,  is  proportional  to  the  resolution  r.  But 
(3.6)  and  (3.9)  combine  to  form 


(3,10) 


Thus  we  cannot  simultaneously  guarantee  fast  execution  and  good 
accuracy:  the  choice  of  r  will  dictate  the  desired  tradeoff 
between  these  two  aspects  of  the  heuristic's  performance. 

If  we  neglect  errors  on  the  order  of  k-digit  roundoff,  (3.10) 
shows  our  method  to  be  a  fully  polynomial  (indeed,  linear) 
approximation  scheme  [1]. 


4.  INTUITIVE  MOTIVATION 

Our  method  was  inspired  by  the  concept  of  bandwidth  as 
a  measure  of  the  inherent  complexity  of  a  function,  and  the 
notion  that  by  limiting  bandwidth,  essential  form  may  be  retained 
while  complexity  is  reduced.  Modern  communications  systems 
utilize  this  technique  to  transmit  approximate  signals  that 

require  only  a  fraction  of  a  line's  transmitting  capacity:  in 

this  way,  thousands  of  signals  may  be  simultaneously  transmitted 
on  a  single  line. 

The  concepts  of  bandwidth  and  transform  are 
closely  related.  Any  signal  may  be  viewed  as  a  linear 
combination  of  sinusoidal  components.  A  transform  expresses  the 
magnitude  of  each  component,  and  the  bandwidth  is  the  range  of 
component  frequencies  contained  in  the  signal.  A  limited- 
bandwidth  signal  approximation  is  obtained  by  erasing  sections  of 
the  transform  of  (intuitively,  removing  part  of  the  complexity 
in)  the  original  signal.  If  we  eliminate  the  components  whose 
frequencies  are  too  small  or  too  large  to  be  of  practical 

concern,  we  are  left  with  a  finite  band  of  frequencies  that 

express  the  signal's  essential  characteristics. 

In  computing  tail  probabilities  from  transforms,  we  view 
the  probability  density  as  a  signal  to  be  approximated.  Every 
evaluation  of  the  transform  yields  information  about  this 
density,  and  we  wish  to  make  do  with  as  few  evaluations  as 
possible.  So  we  eliminate  low-frequency  components  by 
neglecting  the  density  outside  the  range  L  to  U,  and  we  eliminate 
high-frequency  components  by  permitting  A  to  be  displaced  by  a 
small  multiple  of  0.  More  specifically,  restricting  the 
probability  distribution  to  the  range  [L,  U]  restricts  its 
frequency  components  to  integer  multiples  of  w.  and  adding  -DZ 
to  X  multiplies  the  transform  of  X  by  a  transform  whose  values 
decrease  rapidly  as  the  frequency  increases  beyond  1/D.  We  are 
left  with  an  approximate  density  whose  tail  probability  is 
easily  computed. 
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5.  COMPLEXITY  ANALYSIS 


We  now  establish  the  difficulty  of  computing  tail  probabi¬ 
lities  exactly,  even  from  very  simple  transforms.  We  for¬ 
malize  the  problem  as  follows:  Evaluate  P[X>A]  given  a  number  A 
and  a  transform  expressed  in  closed  form,  that  is,  as 

a  finite  word  generated  (according  to  the  grammar  of  complex 
arithmetic)  by  real  numbers  and  the  symbols  s,  e,  -,  »,  /, 
T  ,  (  ,  and  ) . 

We  show  that  this  problem  is  at  least  as  difficult  (in  the 
worst  case)  as  the  following  knapsack  problem:  Given  M-^1 

positive  integers  t. . and  A.  is  there  a  subset  S  of 

sich  that  2t, -A?  This  problem  is  known  to  be 
les  * 

NP-complete  [1],  Hence, 

Theorem  2.  Computing  tail  probabilities  from  transforms  is 
NP-hard. 

Proof.  Let  X  ■  2  with  the  X,  mutually  independent 

and 

{tj,  with  probability  4  , 

0,  with  probability  j  . 

If  we  could  quickly  compute  P{X>A-1J  and  P[X>AJ,  then  we  could 
also  quickly  compute 

P[X>A-1]  -  P(X>A]  =  P[X-A]  -  2~**  •  xr  ^Sc{l,...M)  |  S^t,  -  A^ 

to  see  whether  it  is  non-zero,  and  so  determine  whether  the 

knapsack  problem  has  a  solution.  But  the  knapsack  problem  is  NP- 

hard,  so  the  computation  of  tail  probabilities  must  be  NP-hard  as 


In  this  sense,  the  computation  of  tail  probabilities  from 
transforms  resembles  the  knapsack  problem,  for  which  fully  poly¬ 
nomial  approximation  schemes  are  also  available  [Lawler  (1979)]. 
Note,  however,  that  our  definition  of  the  error  Eg  refers  to  a 
perturbation  of  the  parameter  A.  whereas  the  more  common  defini¬ 
tion  of  error  refers  to  a  perturbation  of  the  result,  t.  This 
distinction  is  necessary  because  we  are  dealing  with  real  num¬ 
bers,  not  integers,  and  because  the  tail  function  P[X>A]  may  be 
discontinuous. 

Like  many  complexity  results.  Theorem  2  states  that  some 
instances  of  a  problem  are  hard  to  solve.  Of  course,  other 
instances  are  easy  to  solve,  for  example,  problems  involving 
transforms  of  uniformly  distributed  random  variables.  It  is 
important  therefore  to  consider  whether  there  are  large 
subclasses  of  problems  for  which  the  conclusion  of  Theorem  2 
fails  to  hold. 

The  proof  of  Theorem  2  rests  on  a  connection  between  certain 
discrete  random  variables  and  the  knapsack  problem.  When  X  is 
discrete,  the  tail  probability  P[X>A1  is  discontinuous  in  A,  and 
this  obviously  makes  the  task  of  computing  tail  probabilities 
more  -difficult.  One  might  be  tempted  to  conclude  that  disconti¬ 
nuity  alone  is  responsible  for  the  conclusion  of  Theorem  2,  i.e., 
that  only  the  tail  probabilities  of  discrete  random  variables 
are  difficult  to  obtain.  But  this  is  not  so-.  The  random 
variables  whose  tail  probabilities  are  sought  can  be  restricted 
to  have  bounded  density,  yet  the  result  of  Theorem  2  still  holds: 
simp'”  redefine  the  random  variables  Xj  in  the  proof  to  be 
uniformly  distributed  over  (0,  1/M]  U  [  t, ,  tj  +  l/M].  Similarly, 
the  conclusion  of  Theorem  2  can  be  shown  to  hold  even  if  the 
density  is  required  to  be  continuous,  or  continuously 
differentiable. 

Thus,  the  class  of  transforms  from  which  tail  probabilities 
are  hard  to  obtain  is  nontrivial,  and  Theorem  2  implies  (under 
the  customary  complexity  assumption  P^NP)  that  no  procedure 
exists  that  will  quickly  compute  tail  probabilities  from  general 
transform  specifications.  This  is  strong  circumstantial  evidence 
that  there  can  be  no  significantly  better  method  than  the  one  we 
have  proposed. 


6.  NUMERICAL  EXAMPLES 


We  now  provide  numerical  examples  to  illustrate  the  advan¬ 
tages  of  our  approach.  Four  random  variables  will  be  considered. 
In  each  case,  we  will  seek  to  determine  the  tail  probability  that 
X  exceeds  A  =  p  4  o,  where  p  and  o  denote  the  mean  and 
standard  deviation  (respectively)  of  X.  These  random  variables 
are  as  described  below.  A  summary  of  problem  parameters  is  given 
in  Table  1.  Note  that  each  of  the  transforms  shown  in  Table  1 
satisfies  the  assumption  (A. 2')  defined  in  Section  2. 

PROBLEM  A.  X^  is  a  normal  random  variable  of  zero  mean 
and  unit  variance.  We  seek  to  compute  4(4).  The  exact 
solution  is  available  in  “probability  tables.”  However,  it 
should  be  noted  that  no  closed-form  expression  for  the  integral 
(3.5)  exists,  and  such  tables  are  not  easy  to  compute! 

PROBLEM  B.  Xg  is  a  binomial  random  variable  representing 
the  number  of  heads  obtained  in  100  flips  of  a  fair  coin.  We 
seek  the  probability  of  more  than  70  heads.  The  exact  solution 
is  obtained  by  summing  the  discrete  probabilities  of  71  through 
100  heads.  As  predicted  by  the  Central  Limit  Theorem,  the  solu¬ 
tion  to  problem  B  is  close  that  of  problem  A. 

PROBLEM  C.  X(.  denotes  the  total  earnings  from  twenty 
games,  where  the  probability  of  winning  the  m-th  game  is 
IZ-vlm,  and  the  winning  payoff  is  -Jm.  (A  losing  game 
earns  nothing.)  W  seek  the  probability  of  winning  more  than 
$51.41  .  The  exact  solution  is  obtained  by  enumerating  outcor.ies. 

PROBLEM  D.  Xp  denotes  the  time  from  arrival  until  service 
begins  for  a  customer  in  an  M/G/1  queue  characterized  by  an 
arrival  rate  of  one  per  minute,  and  a  service  duration  of  40 
seconds  exactly  in  SOX  of  the  cases,  and  2  minutes  exactly  for 
the  remaining  lOX.  (Service  times  are  assumed  to  be  mutually 
independent.)  The  transform,  mean  and  variance  of  Xp  are 
available  from  the  P -K  formula  of  queueing  theory.  We  seek  the 
probability  that  a  customer  must  wait  longer  than  11.61  minutes. 
No  method  (other  than  transform  inversion)  is  available  to  com¬ 
pute  the  exact  solution.  In  problems  of  this  kind,  simulation  is 
often  attempted,  but  simulation,  by  its  very  nature,  cannot 
efficiently  provide  good  estimates  of  small  probabilities.  The 
solution  shown  in  Table  1  was  obtained  by  the  method  given  in 
this  paper. 

The  values  of  U  and  L  in  Table  1  were  chosen  to  span  twice 
the  effective  range  of  each  random  variable.  In  problem  D,  the 
effective  range  was  determined  by  quickly  estimating  tail 
probabilities  with  high  precision  but  low  accuracy  to  show  that 
P  I  Xp  >  3  0  )  <  <  1  , 


Figure  3  shows  the  re lati o r\s hi p  between  accuracy  error 
and  the  approximate  tail  probability,  r,  obtained  when  Ep  and 
Et  are  vanishingly  small.  Note  that,  for  each  problem,  t  had 

come  close  to  its  limiting  value  by  the  time  E,  reached  ~  0.3  . 

The  rate  of  convergence  depends  on  the  smoothness  of  the 

tail  probability  function  -  see  Figure  1.  For  Problems  A,  C  and 
D,  T  approached  the  true  tail  probability  (as  E3-»0).  In  the 

case  of  Problem  B,  it  approached  P[X>701  *  0.5  P[X=70],  which  is 
not  strictly  correct,  but  satisfies  (3.3)  for  any  Note 

that  the  limiting  values  for  Problems  A  and  B  nearly  coincide,  as 
predicted  by  the  Central  Limit  Theorem,  but  that  B  converges  more 
slowly,  presumably  because  of  the  discontinuities  in  its  tail 
probability  function. 

Figure  4  shows  the  truncation  error  |t-0|  as  a  function  of 
the  normalized  computational  effort  z^  -  see  (3.9).  The  bound 
on  truncation  error  (3.8)  is  also  shown  in  the  upper  right  corner 
of  each  graph.  Case  A  converges  the  fastest  -  because  the 
magnitude  of  its  transform  decreases  rapidly  as  the  index  n  in 
(1.3)  increases.  Case  B  remains  constant  over  many  iterations 
before  changing,  and  then  goes  through  a  few  intermediary  stages 
with  relatively  large  error.  This  demonstrates  that  the  sum 
(1.3)  cannot  safely  be  terminated  after  its  terms  have  remained 
small  for  a  long  time,-  they  may  suddenly  increase  periodically! 

Note  particularly  that  t  approaches  0,  not  the  true  tail 
probability,  as  N  increases.  Quick  convergence  in  the  sense  of 
Figure  4  does  not  imply  that  a  good  approximation  has  been 
obtained. 

7.  CONCLUSIONS 

The  problem  of  determining  a  tail  probability  P[X:»A],  given 
only  the  transform  of  X,  has  been  shown  to  be  NP-hard,  implying 
that  a  computationally  efficient  algorithm  to  solve  it  exactly 
probably  does  not  exist.  Instead,  we  propose  to  use  the  simple 
approximation  t  given  by  (1.3)-(1.4).  The  closeness  of  this 
approximation  is  given  by  (3.3).  Eqs.  (3.7)-(3.8)  show  how  the 
"precision"  errors  can  be  made  arbitrarily  small,  and  (3.10) 
expresses  the  tradeoff  between  "accuracy"  errors  and 
computational  effort. 

We  anticipate  that  this  method  will  be  particularly  useful 
when  computational  resources  are  scarce  and  rough  estimates  of 
the  tail  probability  will  suffice,  notably,  when  the  threshold  A 
is  not  exactly  known,  or  when  tail  probabilities  are  required  to 
evaluate  alternatives  in  subroutines  of  large-scale  optimization 
programs.  However,  when  "exact"  solutions  are  called  for.  our 
method  will  generally  perform  at  least  as  well  as,  and  certainly 
more  efficiently  than,  others  currently  available. 
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APPENDIX  A 


DERIVATION  OF  THE  FOURIER  SERIES  (3.2) 


Let 


g(y) 


{1,  if  y-L  mod  U-L  >  A-L  mod  U-L. 
0,  If  y-L  mod  U-L  <  A-L  mod  U-L. 


Since  g(*)  is  periodic,  it  is  given  by  the  exponential 
series 


g(y) 


09 

2 

ns-«» 


nuiyj 


Where 


«(y)  dy  ■  .  n  -  ii,  i2. 

*•»  *  lAr  ■  0^  • 


and  P,  y  and  cj  are  given  by  (1.4).  If  Y  is  any 
variable,  then  by  (A. 2)  and  (1.1), 

E[g(Y)]  -  5  C„  E  C  e""^^  ]  -  2  C„  Sfyl-nwJ). 

n*-«»  ns-«» 

Moreover,  if  Y  is  continuously  distributed,  then  we  may 
the  points  of  discontinuity  of  g(*).  and  (A.l)  implies 


(A.1) 

Fourier 

(A.2) 

(A.3) 

random 

(A.4) 

neglect 


E(g(Y)]  •  Prob{Y-L  mod  U-L  >  A-L  mod  U-LJ. 


(A.5) 


Let  Y  -  X-DZ,  where  X,  D  and  Z  are  as  in  (3.1).  Y  is 
continuously  distributed,  so  (A. 5]  holds.  By  (3.1)  and  (A. 5). 


0  -  E(g(Y)I, 


(A.6) 


and  by  (A. 4)  and  (A.6), 


0  *  2 

ns-«* 


(A.7) 


Since  Y-X-DZ,  (1.2)  implies  £y(s)  -  (A.7) 


becomes 


ns-«o 


Moreover  -DZ  is  normally  distributed  with  mean  0  and  variance 

,  so  S?_o2  ( s )  *  ,  where  a  is  defined  by  (1.4). 

Substituting  this  into  (A. 8)  yields 


0  -  S  C-  «x(-ncaj). 

n  A 


By  (1.1),  if(s*)  -  Sf*(s)  and  by  (A. 3),  C.^-C^*  (where  •  denotes 
complex  conjugate).  So  (A. 9)  may  be  written  as 


0  -  Cn  ♦  2  5  reaKCn  S?x(-nwj)). 


(A.10) 


Substituting  (A. 3)  into  (A. 10)  produces  (3.2). 


APPENDIX  B 


PROOF  OF  THEOREM  1 


By  (3.1), 


e  <  PIX-DZ>A  or  X-DZ<L1  <  P[X>A>DZ]  ♦  P(X<L-DZ1. 


0  >  P[X-DZ2A  and  X-DZ<U1  -  PIX>A*DZJ  -  PlX>U^DZl, 


P[X:>A*DZ1  -  P{X>A-^DZ|Z>-ZpJ  P{Z>-Zp]  ♦  PtX>A^DZ|Z<:-2p]  P(Z<-Zp] 

<  P[X>A*DZ|Z>-Zp]  >  PtZ<-2p] 

<  PtX>A-D2p]  $(2p). 

Proceeding  similarly  to  bound  the  remaining  probabilities  on  the 
right-hand  side  of  (B.l),  we  obtain 

0  <  PCX>A-DZp]  >  P[X<L*DZp]  -  2  4(2^), 

(B.2) 

0  >  PtXaA+DZp]  -  P(X>U-DZp]  -  2  *(Zp). 

Now  (B.2)  and  (3.6)-(3.7)  imply  the  first  part  of  (3.3). 

We  now  turn  our  attention  to  the  second  part  of  (3.3).  By 

(1.1)  and  (1.4), 


||!ifx(-nwj)((  <  1. 


By  (3.2)  and  (1.3), 


HPII  1  1. 


lIVll  £  1. 


CO  2 

'  •  !"■[»"  -  r")  • 


^  ni**!  W  •  (IIP"II 


(by  (B.4)) 


i  «n2 


(by  (B.3)) 


<  Z  o"^ 

Nn  n«N-*-l 


(since  n>N  over  the  range  of  summation) 


^  FhT  ir 


(by  monotonicity  of  the  integrand) 


2.  f*  e-(2iim)2/2  . 

N*  J*.  ® 


(by  (1.4)  and  (3.4)) 


2  .-*2/2 
2nrNa  J2wrN 


(z  *  2nrn) 


S(2  ) 

Z^JI 


(by  (3,5)  and  (3.9)) 


<  £|  . 


(by  (3.8)) 


This  establishes  the  second  part  of  (3.3). 
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Figure  1.  Geometric  interpretation  of  E^,  E  and  E^. 


Figure  3 


□  PROBLEM  A  O  PROBLEM  C 

+  PROBLEM  B  A  PROBLEM  D 


Approximate  tail  probability  T  as  a  function  of 


E  when  z  »z  *7. 
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